Skip to content

Documentation for self-hosting with docker compose #855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 25, 2025

Conversation

unteem
Copy link
Contributor

@unteem unteem commented Apr 7, 2025

Purpose

Make self hosting of Docs easier with an example of a deployment procedure with docker compose and document how to configure Docs.

While #583 propose an easy way to deploy Docs with docker and Make, here we describe more in details the various steps and requirements to deploy Docs.

Proposal

  • example to deploy and configure keycloak
  • example to deploy and configure minio
  • example to configure proxy and certs
  • example to deploy and configure Docs

Improvements

  • Rephrase description of environment variables and categorize
  • Use template for nginx conf

Fixes #561
Supersedes #583

A one liner quick start could be a nice addition:

  • merge all services in a single compose
  • scripts to generate secrets

@unteem unteem force-pushed the documentation-docker branch 3 times, most recently from ae27df7 to 3531bf4 Compare April 8, 2025 08:50
@AntoLC AntoLC requested a review from lunika April 8, 2025 19:23
Copy link
Member

@lunika lunika left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you commits does not have a description. It is useful to add it to explain why you made the modification about the env.md file for example.

@mustafa0x
Copy link

This commit will make docs more accessible to a huge number of teams. Eagerly waiting to see it committed!

@unteem unteem force-pushed the documentation-docker branch 2 times, most recently from 94046a7 to 99bd17d Compare May 20, 2025 13:13
@unteem
Copy link
Contributor Author

unteem commented May 20, 2025

Thanks for the reviews, should be good now.

@lunika I removed the commits that change env.md and will make another PR of those changes make sense. I think its better to split it.

@virgile-dev virgile-dev requested review from qbey and lunika May 20, 2025 15:10
@unteem unteem force-pushed the documentation-docker branch from 99bd17d to b5cd3e7 Compare May 21, 2025 09:05
Copy link
Collaborator

@virgile-dev virgile-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we keep an installation.md at the root of the /docs folder where we can list the different deployment option (K8S, Compose and NIX) ?

Copy link
Collaborator

@virgile-dev virgile-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this and make documentation for different deployment solution more homogeneous in a next PR.

@virgile-dev virgile-dev removed the request for review from qbey May 23, 2025 09:17
@virgile-dev virgile-dev enabled auto-merge (squash) May 23, 2025 09:19
auto-merge was automatically disabled May 23, 2025 10:41

Head branch was pushed to by a user without write access

@unteem unteem force-pushed the documentation-docker branch 7 times, most recently from 072463a to b178c35 Compare May 23, 2025 13:09
@virgile-dev virgile-dev enabled auto-merge (squash) May 23, 2025 20:02
@virgile-dev
Copy link
Collaborator

@unteem can you make the last test pass so we can merge this please ?

@AbdelRahmanAlTamimi
Copy link

@unteem
Hi, I've noticed that the curl commands provided in the docs/installation/compose.md and other files are currently resulting in a 404: Not Found error. Could you please investigate and update the documentation accordingly? Thanks!

OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
#USER_OIDC_FIELD_TO_SHORTNAME
#USER_OIDC_FIELDS_TO_FULLNAME
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've fixed that in #1100


### Additional notes:

For increased security you should create a dedicated user with `readwrite` access to the Bucket. In the following example we will use MinIO root user.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be clarified imo, it took me a while until I made the connection of currently root user & password credentials being the ones for

AWS_S3_ACCESS_KEY_ID=<s3 access key>
AWS_S3_SECRET_ACCESS_KEY=<s3 secret key>

Also maybe at least link to outside docs that explain how to set up a user? I imagine most (including me) are not familiar with how one usually sets up minio.

I'm currently trying to set that up, from my understanding I need to create a new user, https://min.io/docs/minio/linux/administration/identity-access-management/minio-user-management.html#id1 and then it sounds like I shouldn't use the user credentials directly but create an accesskey to put into the config? https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-accesskey-create.html#command-mc.admin.accesskey.create

Copy link
Contributor Author

@unteem unteem Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I added an example but I eventually dropped it. The intention here is more to provide a quick start example than a thorough process to go to production. I don't know if we want to go that deep in details, like creating a user with the correct permissions.

Docs requires some "complex" parts like an IDP and Object Storage, I think its important for those who want to self host those parts to read the upstream doc and understand by themselves all the implications.

In this context I think skipping user and policy creation and using the minio root user is good enough as an example.

In this case to create the user it would look like this:

  1. Create policy for the bucket
cat > readwritedocs.json << EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::<your-bucket-name>/*"
            ]
        }
    ]
}
EOF
mc admin policy create myminio readwritedocs readwritedocs.json
  1. Create user
mc admin user add myminio <username> <password>
  1. Assign policy to user
mc admin policy attach myminio getonly --user=<your-user>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious to have your thoughts on this @virgile-dev
should I add this part in the example ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lunika What do you think ?

Copy link
Member

@lunika lunika Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A warning can be added on the OIDC and Minio part to make sure people read the related project documentation on how to go in production with these tools.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unteem can you update accordingly please ?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unteem Thanks for sharing the policy stuff!

And I guess the step after is to run mc admin accesskey create myalias <username> --access-key $AWS_S3_ACCESS_KEY_ID --secret-key $AWS_S3_SECRET_ACCESS_KEY, right? It's a bit difficult to figure out what Minio recommends for such as setup, e.g. you can also use the Minio root user & password for AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY and that works totally fine. I assume you can also use the user credentials we just created and that'll work. But given the variable naming in docs, I assume creating an accesskey is the recommended thing to do.

From my perspective, I'm setting this up as a small instance for at most 20 people that use it occasionally, so we don't need any fancy big setup with sharding, load balancing, etc. We also only have everything running on one server, so there's zero high-availability anyways.

So having in these docs here how to get to a usable and secure production (while ignoring all HA and extra things for larger instances) would be good imo.

Thanks again for working on this!

Copy link
Contributor Author

@unteem unteem Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A warning can be added on the OIDC and Minio part to make sure people read the related project documentation on how to go in production with these tools.

Its already the case, there are 3 warnings in the beginning pointing to upstream documentation

> \[!CAUTION\]
> We provide those instructions as an example, it should not be run in production. For production environments, deploy MinIO [in a Multi-Node Multi-Drive (Distributed)](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#minio-mnmd) topology

> \[!CAUTION\]
> We provide those instructions as an example, for production environments, you should follow the [official documentation](https://www.keycloak.org/documentation).

On minio I also point out that in prod, minio root user should not be used but a user with readwrite access

I'm not sure I can do more ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unteem Thanks for sharing the policy stuff!

And I guess the step after is to run mc admin accesskey create myalias <username> --access-key $AWS_S3_ACCESS_KEY_ID --secret-key $AWS_S3_SECRET_ACCESS_KEY, right? It's a bit difficult to figure out what Minio recommends for such as setup, e.g. you can also use the Minio root user & password for AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY and that works totally fine. I assume you can also use the user credentials we just created and that'll work. But given the variable naming in docs, I assume creating an accesskey is the recommended thing to do.

From my perspective, I'm setting this up as a small instance for at most 20 people that use it occasionally, so we don't need any fancy big setup with sharding, load balancing, etc. We also only have everything running on one server, so there's zero high-availability anyways.

So having in these docs here how to get to a usable and secure production (while ignoring all HA and extra things for larger instances) would be good imo.

Thanks again for working on this!

Yep if you want to use mc to do operations like creating a user, CRUD on buckets/objects etc. you will need to set an alias

I would recommend to use a specific user and not the root/admin even if in your setup it could be fine to use the admin user, its always better to have least privileges access. It makes even more sense if at one point you use this minio instance for other apps, better to have one "user" per app.

auto-merge was automatically disabled June 3, 2025 11:33

Head branch was pushed to by a user without write access

@unteem unteem force-pushed the documentation-docker branch 2 times, most recently from 3ebf76b to 2735a56 Compare June 3, 2025 12:22
@unteem unteem force-pushed the documentation-docker branch 2 times, most recently from 3a9db6d to 822e1ee Compare June 13, 2025 08:25
@AbdelRahmanAlTamimi
Copy link

@unteem

Hi, thank you for the update!
Just a quick note — the curl commands in this section return a 404 error:
https://github.com/unteem/docs/blob/documentation-docker/docs/installation/compose.md#step-1-prepare-your-working-environment

It seems the URLs might be outdated or incorrect. Could you please double-check them?

Thanks again!

- ./data/databases/backend:/var/lib/postgresql/data/pgdata

redis:
image: redis:5
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I mistaken or is Redis 5 EOL since a long time?

https://redis.io/docs/latest/operate/rs/installing-upgrading/product-lifecycle/

Given the new license of Redis it might also make sense to use Valkey by default instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch update to v8, I don't use redis in prod... :)
I think as an example it should be fine to leave redis, Valkey is indeed a good option.

@unteem
Copy link
Contributor Author

unteem commented Jun 25, 2025

@unteem

Hi, thank you for the update! Just a quick note — the curl commands in this section return a 404 error: https://github.com/unteem/docs/blob/documentation-docker/docs/installation/compose.md#step-1-prepare-your-working-environment

It seems the URLs might be outdated or incorrect. Could you please double-check them?

Thanks again!

URL are broken because they are pointing to the final URL, as the PR is not merged yet you need to replace https://raw.githubusercontent.com/suitenumerique/ by https://raw.githubusercontent.com/unteem/

@unteem unteem force-pushed the documentation-docker branch from 822e1ee to d99c508 Compare June 25, 2025 11:45
@unteem
Copy link
Contributor Author

unteem commented Jun 25, 2025

@unteem
Hi, thank you for the update! Just a quick note — the curl commands in this section return a 404 error: https://github.com/unteem/docs/blob/documentation-docker/docs/installation/compose.md#step-1-prepare-your-working-environment
It seems the URLs might be outdated or incorrect. Could you please double-check them?
Thanks again!

URL are broken because they are pointing to the final URL, as the PR is not merged yet you need to replace https://raw.githubusercontent.com/suitenumerique/ by https://raw.githubusercontent.com/unteem/

Ok some were broken 🙈 and you need to replace https://raw.githubusercontent.com/suitenumerique/docs/refs/heads/main/ by https://raw.githubusercontent.com/unteem/docs/refs/heads/documentation-docker/ (with my branch).

Its fixed now

@virgile-dev virgile-dev enabled auto-merge (squash) June 25, 2025 13:00
@virgile-dev virgile-dev merged commit 1e76e6e into suitenumerique:main Jun 25, 2025
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add docker compose up production configuration
8 participants